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1. Introduction 

. There have been many attempts to give a coherent formulation of a hier- 

f\| | archical progression that would lead to a refined partition of the vast area 

stretching from the context-free to the context-sensitive languages. The pur- 
pose of this note is to describe a theory that seems to afford a promising 
' method of interpreting the tree adjoining languages as the natural third step 

in a hierarchy that starts with the regular and the context-free languages. 
The formulation of this theory is inspired by two sources: the categori- 
}^ ' cal concept of an algebraic theory and the powerful tool of macro variables 

t^- ■ which is well established within the framework of program schemes. The 

rough idea is that according to their intended interpretation objects of alge- 
bi)' braic theories are sets of derived operations and that macro variables range 

over these sets. Guided by this conception we show how monadic macro vari- 
ables provide a perspective from which both the context-free and the tree 
adjoining languages become realizations of the same general notion when 
the relevant underlying algebra is specified. The context-free languages are 
determined by an inductive process in which monadic macro variables are 
^ . replaced by derived operations of an algebra all of whose operations are 

| unary. In the case of the tree adjoining languages the same substitution 

process is applied to derived operations over an arbitrary algebra. 

The central notion in this account is that of a higher-order substitution. 
Whereas in traditional presentations of rule systems for language families the 
emphasis has been on a first-order substitution process in which auxiliary 
variables are replaced by elements of the carrier of the proper algebra — 
concatenations of terminal and auxiliary category symbols in the string 
case — we lift this process to the level of operations defined on the elements 
of the carrier of the algebra. Our own view is that this change of emphasis 
provides the adequate platform for a better understanding of the operation 
of adjunction. To put it in a nutshell: Adjoining is not a first-order, but a 
second-order substitution operation. 

This is not the first place that macro productions are put to use outside 
the field of program schemes. It has been known since the pioneering work of 
M.J. Fischer that macro grammars (without the restriction to monadic vari- 
ables) are weakly equivalent to indexed grammars. The paper by Maibaum 
also contains one direction of the equivalence between context-free grammars 
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and monadic extensions of regular grammars. The other direction seems to 
be part of the folklore wisdom pertaining to tree language theory. 

The greatest advantage that comes with a macro-like presentation derives 
from an explicit notation for derived operators in an algebra. As was em- 
phasized above, macro variables range over complex or derived operations. 
These derived operations are formed by functional composition, starting 
from the primitive operations. There is an alternative conception of this 
compositionally closed family of functions as an algebra whose carrier is the 
disjoint union of the derived operations and whose only primitive operations 
are just the suitably typed composition functions. This conception views op- 
erations as elements of a many-sorted algebra. The corresponding shift on 
the symbolic level " nominalizes" the second-order macro variables and maps 
them into object variables. As a result of this translational shift a system 
with macro-like productions becomes an instance of a regular tree gram- 
mar. This latter grammar format provides the class of structures that fall 
under the purview of logical techniques that have been successfully applied 
to problems surrounding the metatheory of several linguistic models. The 
algebraic approach sketched above will enable us to apply these techniques 
to structural properties of grammatical formalisms that properly include the 
range of mildly context-sensitive devices. 

In the following we will first introduce the basic notion of a context-free 
tree grammar (Section 2). Then, in Section 3, the equivalence between 
context-free string languages and context-free tree languages over monadic 
alphabets is established. Section 4 is devoted to the presentation of the tree 
adjoining languages as languages generated by context-free tree grammars 
where all the functional symbols are monadic. The concluding section points 
to the model theoretic techniques that are applicable to the kinds of tree 
producing systems underlying the constructions in this paper. 

2. Preliminaries 

The purpose of this section is to fix notations and to present definitions 
related to tree grammars. 

Definition 1. A ranked alphabet or signature E is an indexed family 
S n (n £ N) of disjoint sets. A symbol in E n is called an operator of rank n. 
If n = then / G So is also called a constant. 

Example 1. 

a) S = {e} U V S 2 = {-} 

Single-sorted signature of semi-groups, extended by a finite set of con- 
stants V. 

b) S = {e} Si = {a | a € V} 
Single-sorted signature of a monadic algebra. 

Definition 2. For a ranked alphabet S, we denote by T(S) the set of trees 
over S. T(S) is inductively defined as follows: 

(i) S C T(S) 

(ii) If / € E„ and U e T(S) for i = 1, . . . , n 
f(h,...,t n ) €T(£) 
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We fix an indexed set X = {x±,X2, . . . } of variables and denote by X n the 
subset {x±, . . . , Xn}. Variables are considered to be constants, i.e. operators 
of rank 0. For a ranked alphabet £ the family T(£, X) is defined to be 
T(£(X)), where £(-X") is the ranked alphabet with T,(X)o = So U X and 
T,(X) n = S n for every n^O. For Y C X a tree i G T(£, X) is called linear 
in Y" if no variable in Y has more than one occurrence in t. A subset L of 
T(£) is called a iree language over S. 

Having described the tree terms, it remains to specify the central notion 
of an algebra and to give a precise definition of the way in which the operator 
symbols induce an operation on an algebra. 

Definition 3. Suppose that £ is a ranked alphabet. A £-algebra A is a 
pair A = (A, (/a)/ge) where the set A is the domain or carrier of the 
algebra and for each operator / G £ n , /a : A n — > A is an operation of arity 
n on A. 

Different algebras, defined over the same operator domain, are related to 
each other if there exists a mapping between their carriers that is compatible 
with the basic structural operations. 

Definition 4. A S-homomorphism of £-algebras h : A — ► B is a func- 
tion h : A — ► B, such that for every operator / of rank n 

^(/a(«i, • • • , a n )) = /b(^(oi), • • • , h(a n )) 
for every re-tuple (a±,..., a n ) G A n . 

The set of trees T(T,,X) can be made into a S-algebra by defining the 
operations in the following way. For every / in S n , for every (t\, • • • , tn) in 
T(S,X) n 

/t(E,X)(*1> ■ ■ ■ >*n) = • • • ,t n )- 

Every variable-free tree t € T(S) has a value in every S-algebra A. It is 
the value at t of the unique homomorphism h : T(S) — > A. 

The existence of a unique homomorphism from the S-algebra of trees into 
an arbitrary S-algebra A provides also the basis for the view that regards 
the elements of T(£, X n ) as derived operations. Each tree t G T(E, X n ) 
induces an n-ary function 

t A : A™ -> A 

The meaning of this function is defined in the following way. For every 
(ai,... ,a n ) G ^4™ tA(ai,... , a n ) = a(t), where a : T(£,X n ) — > A is the 
unique homomorphism with a(xj) = dj. 

In the particular case where A is the S-algebra T(S, X m ) of trees over S 
that contain at most variables from X m = {x±, . . . ,x m } at their leaves the 
unique homomorphism extending the assignment of a tree ij G T(E, X m ) to 
the variable Xj in X n acts substitution: 

*T(E,X m )(*l> ■■■,*«) = *[*1> •••,*«] 

where the last tree indicates the result of substituting tj for Xi in t. 

Let F be an ordinary finite alphabet. It gives rise to a monadic signature 
Sy if all the members of V are assigned rank one and a new symbol e 
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is added as the single constant of rank zero. As was pointed out in the 
previous paragraph, there is a unique homomorphism from the trees over £y, 
considered as the carrier of a Ey-algebra, to any other algebra of the same 
type. In particular, there is a homomorphism into V* when every a £ V, i.e. 
a € (£y)i, is interpreted as left-concatenation with the symbol a and when 
£ is interpreted as the constant string of length zero. This homomorphism is 
actually an isomorphism between T(£y) and V* (cf. [Maibaum 1974 ). This 



isomorphism will play an important role in the next section on context-free 
string languages. 

We now turn to introducing the notion of a context-free tree grammar 
(cftg). This type of grammar is related to the type of grammars that were 
defined by [Fischer (1968 ) and were called macro grammars by him. Context- 



free tree grammars constitute an algebraic generalization of macro grammars 
since the use of macro-like productions served the purpose of making simul- 
taneous string copying a primitive operation. 

Definition 5. A context free tree grammar G = (£, !F, S, P) is a 4- 

tuple, where £ is a finite ranked alphabet of terminals, is a finite ranked 
alphabet of nonterminals disjoint from S, S € J- is the start symbol of rank 
and P is a finite set of rules of the form 

F(xi, . . . ,x m ) -> t (n > 0) 

where F G J- m and t £ T(S U J-, X m ). Recall that X is here assumed to be 
a set of variables X = {x±, x%, . . . } and X m = {x\, . . . , x m }. 

In the conventional case of context-free string grammars the class of lan- 
guages generated by unrestricted derivations is identical to the class of lan- 
guages generated by left-most derivations (or by right-most derivations, for 
that matter). Such an equivalence cannot be proved in the tree case. While 
this is true for cftg's in general, the difference between the derivation modes 
has no effect on the special category of monadic linear tree grammars, which 
will be our only concern in the sequel. 

The following definition records the unrestricted derivation mode. 

Definition 6. Let G = (£, J 7 , S, P) be a cftg and let t, t' G T(S UT). t' is 
directly derivable from t (t => t') if there is a tree to € T(S U X{) con- 
taining exactly one occurrence of x\, a corresponding rule F(xi, ■ ■ ■ , x m ) — > 
t", and trees t±, . . . , t m € T(S) such that 

t = tolFfa, . . . ,tm)] 

t' = to[t' [ti, . . . , t m ]] 

t' is obtained from t by replacing an occurrence of a subtree F{t\, . . . ,t m ) 
by the tree t"[ti, . . . , t m ]. 

Recall from above that for m, n > 0, t € T(E,X m ) and ti,...,t m € 
T(£, X n ) t[t\, . . . , t m ] denotes the result of substituting t{ for X{ in t. Ob- 
serve that t\t\, . . . , t m ] is in T(£, X n ). 

As is customary denotes the transitive-reflexive closure of =>. 

Definition 7. Suppose G = (T,,J=;S,P) is a cftg. We call 

C(G,S) = {te T(E)|S4 t} 
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the context-free tree language generated by G from S. 

We reserve a special definition for the case where T contains only function 
symbols of rank zero. 

Definition 8. A regular tree grammar is a tuple G = (S,^ 7 , S, P), 
where E is a finite ranked alphabet of terminals, T is a finite alphabet 
of function or nonterminal symbols of rank zero, S E J- is the start symbol 
and P C T x T(E U T) is a finite set of productions. The regular tree 
language generated by G is 

£ = {te T(E) | 5 4> i} 

Note that in the case of regular grammars the analogy with the con- 
ventional string theory goes through. There is an equivalence of the unre- 
stricted, the rightmost and the leftmost derivation modes where the terms 
'rightmost' and 'leftmost' are to be understood with respect to the linear 
order of the leaves forming the frontier of a tree in a derivation step. 

3. Monadic Trees 

Let us view grammars as a mechanism in which local transformations on 
trees can be performed in a precise way. The central ingredient of a grammar 
is a finite set of productions, where each production is a pair of trees. Such a 
set of productions determines a binary relation on trees such that two trees t 
and t' stand in that relation if t' is the result of removing in t an occurrence 
of a first component in a production pair and replacing it by the second 
component of the same pair. The simplest type of such a replacement is 
defined by a production that specifies the substitution of a single-node tree 
to by another tree t±. Two trees t and t' satisfy the relation determined by 
this simple production if the tree t' differs from the tree t in having a subtree 
t\ that is rooted at an occurrence of a leaf node to in t. In slightly differ- 
ent terminology, productions of this kind incorporate instructions to rewrite 
auxiliary variables as a complex symbol that, autonomously, stands for an 
element of a tree algebra. As long as the carrier of a tree algebra is made 
of constant tree terms the process whereby miliary variables are replaced by 
trees is analogous to what happens in string languages when a nonterminal 
auxiliary symbol is rewritten as a string of terminal and non-terminal sym- 
bols, independently of the context in which it occurs. The situation changes 
dramatically if the carrier of the algebra is made of symbolic counterparts 
of derived operations and the variables in production rules range over such 
second- level entities. The following example illustrates the gain in gener- 
ative power to be expected from production systems determining relations 
among trees that derive from second-order substitutions of operators rather 
than constants. 

Example 2. Let us assume that we are dealing with a vocabulary V that 
contains the symbols a and b as its only members. Trees over the associ- 
ated monadic signature E = So U Ei, where So = {e} and Ei = {a,b} are 
arbitrary sequences of applications of the operators a and b to the constant 
e. As will be recalled from the preceding section, trees in T(E, X) can be 
regarded as derived operators. Due to the fact that E is a monadic signature 
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these trees may not contain more than a single variable. Auxiliary symbols 
ranging over this domain take therefore as values monadic or constant de- 
rived operators. When monadic auxiliary symbols appear in productions 
this means that they behave the way that nullary auxiliary symbols do ex- 
cept for the fact that their argument has to be inserted into the unique 
variable slot of their replacing derived operator. After these explanations it 
should be obvious that the following grammar over the monadic signature £ 
generates the context-free string language {a n b n } when combined with the 
unique isomorphism mentioned above: 



(Z,F,S,P) 

ft = {F} 
F(e)\e, \ 
a(F(b(x)))\a(b(x)) J 

n n 

C{G,S) = {a{^.^biy^.{e)...)} 

The grammar in the last example illustrates for the language L = {a n b n } 
a transformation that can be applied to the grammar of any given context- 
free language. 

Claim 1. For every finite alphabet V, the context-free tree subsets of Tilly) 
(where Y>y is the monadic signature induced byV) have exactly the context- 
free string languages as values of the unique isomorphism that maps trees 
in T(T,y) into strings in V* and interprets the elements of V as unary 
operations that concatenate for the left with their argument. 

Proof. Let G = (A, V, S, P) be an arbitrary context-free string grammar, 
where A is a finite set of nonterminal symbols, V the finite terminal alphabet 
disjoint from N,S G A the initial or start symbol and P C A x (N U V)* 
a finite set of productions. We associate with G an equivalent context-free 
tree grammar G' = (Ey,P, S, P') in the following way. We let £y be the 
monadic ranked alphabet corresponding to V. We let P = PqU P\ where 
Po = {S'} and P\ = N, be the ranked alphabet of nonterminals. The new 
set of productions P' is obtained as follows. If A — ► w± . . . w n is in P, where 
w i G A U V then A(x) — > W\(w2{- ■ ■ (w n (x) . . . ) is in P' . In addition, we 
have a rule for the new start symbol: S' — > S(e). It should be clear from 
the preceding example that L(G) = h(L(G')) where h denotes the unique 
isomorphism between V* and T(Y>y). For the other direction assume that 
G = (S, P, S, P) is an arbitrary cftg over the monadic terminal alphabet 
S. We associate with G an equivalent context-free string grammar G' = 
(P, S, S, P') in the following way. P and E are regarded as unranked sets 
and play the role of nonterminal and terminal alphabets, respectively. The 
set of productions P' is obatined from the paths in the members of P. More 
precisely. Let F{x\, . . . , x n ) — > t be a member of P with t G T(S U P, X n ). 
Then each F — > t' is in P', wher t' is the concatenation of symbols from £ 
and P that label a path through t. It should be obvious that nonterminals 
in P n for n > 1 have to be ultimately replaced by linear or constant monadic 
trees in a derivation according to the original cftg G if they are to contribute 



G = 

Po = {S} 
S 

F(x) - 
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to the generated language at all. Spurious nonterminals are retained in the 
process of substituting the path sets for the trees on the right hand sides of 
the rules in P, but they still do not affect the string languages specified by 
G' . Indeed, under the unique isomorphism between V* and T(E) it holds 
that G and G' generate the same language. □ 

Let V be an ordinary finite alphabet and Sy ist associated monadic signa- 
ture. Recall that all nonterminals in a regular tree grammar are of arity zero. 
An arbitrary rule in a regular tree grammar over the monadic signature Sy 
can therefore assume only one of the following two forms: 

A ^a 1 (a 2 {. . . a n (e) ...) 
A -mi(a 2 (. . . a n (B) ...) 

where a% € (£y)i, i.e. the ctj's are the monadic terminal operators corre- 
sponding to the members of V and A and B are two nonterminals of rank 
zero. Under the unique isomorphism that relates strings with monadic "ver- 
tical" trees the two rule formats above correspond exactly to the two types of 
rules in regular string grammars. This implies that a language L generated 
by a regular tree grammar over a unary ranked alphabet is, interpreted as a 
string language, the language generated by the corresponding regular string 
grammar. Based on the claim about the relationship between context-free 
string grammars and context-free tree grammars over a finite vocabulary and 
its associated monadic signature, respectively, we have therefore established 
the result, announced in the introduction, that the increase in generative 
power which characterizes the shift from the regular to the context-free lan- 
guages, is due to the transgression from a miliary, first-order substitution 
process to its higher-order analogue. 

4. Adjoining Trees 

Very early in the development of (regular) tree grammars it was realized 
that there exists a close relationship between the families of trees generated 
by tree grammars and the family of context-free string languages. This 
fundamental fact is best described by looking at it from the perspective 
on trees that views them as symbolic representations of values in arbitrary 
domains. 

Definition 9. Suppose S is a ranked alphabet. We call yield or frontier 
the unique homomorphism y that interprets every operator in S n as the 
n-ary operation of concatenation. More precisely 

V(f) = f for /G S 

y(/(ti,...,t„)) = y{h)...y(t n ) for / e S n and U G T(E) 

Fact A (string) language is context-free iff it is the yield of a regular tree 
language. 

As was shown in the last section, the addition of macro variables or 
n-ary nonterminals (n > 0) increases the generative power of cftg's over 
monadic alphabets considerably. In addition, the transformation of an arbi- 
trary context-free tree grammar over a monadic alphabet into an equivalent 
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context-free string grammar has shown that it is only the monadic nontermi- 
nals that are operative in the derivation process with a terminal outcome, i.e. 
a (tree-) expression over the terminal alphabet. The obvious question that 
presents itself in this situation is whether the addition of monadic nonter- 
minals to a regular grammar over an arbitrary signature leads to a langugae 
family that has already been introduced for independent reasons. 

Example 3. Let G = (Ti,J-,S',P) be a context-free tree grammar with the 
following specifications: 

£ = Eo U S3 where £0 = { a > b, c, d} and S3 = {S} 

T = To U T\ where T§ = {S } and T\ = {S} 

P = {S' -» S(e), S(x) -» x, S(x) -» S(a, S(S(b, x, c)), d)} 

In tree form the last rule has the following shape: 

S 



S(x) — > a S 

S 



b x c 

This grammar generates the string language {a n b n c n d n }, i.e., y(L(G)) = 
{a n b n c n d n }. 

Apart from minor notational modifications the grammar in the last exam- 
ple corresponds to a tree adjoining grammar which serves as an illustration 
in the paper by Vijay-Shankcr and Weir (1992| ). Note that apart from the 



start symbol the only other nonterminal is of arity one. As was the case 
in connection with the context-free string languages, the preceding example 
is a particular instance of the general situation. The tree adjoining lan- 
guages correspond to the context-free tree languages that are specified by 
rule systems with nonterminals of arity at most one. 

We shall prove the equivalence between tree adjoining languages and 
context-free tree languages with a monadic rule system for the case of tree 
adjoining grammars with a restricted set of adjunction constraints. 

The device of adjunction constraints was introduced by A. Joshi for the 
purpose of specifying which auxiliary trees can be adjoined at a given node. 
The distinction in cftg's between functional nonterminal and terminal la- 
bels corresponds to the distinction between nonterminals with an obligatory 
and a null adjunction constraint in an initial or an auxiliary tree of a tree 
adjoining grammar. In the theorem below we shall cover this particular 
distinction. The proof can be easily adapted to the general case. 

Definition 10. A tree adjoining grammar (tag) G is a quintuple G = 
(V, N, S, I, A), where (i) V is a finite set of terminal symbols, (ii) A is a finite 
set of nonterminal symbols disjoint from V, (iii) S € N is the distinguished 
start symbol, (iv) I is a finite set of trees, called initial trees, whose root 
label is S and (v) A is a finite set of trees, called auxiliary trees, that have a 
distinguished frontier node, called foot node, whose label is identical to the 
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label of the root node. Only frontier nodes in I or A are labelled by terminal 
symbols. Nonterminal symbols can label both frontier and interior nodes. 
The case where a node is specified by an obligatory adjunction constraint is 
indicated by a bar over the node label. 

Definition 11. Let G = (V, iV, S, I, A) be a tag and let t,t' be two trees 
with node labels from V and N. t' is directly derivable from t (t => t') if 
there is a tree to containing exactly one occurrence of a new leaf label £, an 
auxiliary tree t A with root and foot node label A such that 

t =t (t"/() 

t' =to(t A (f/A)/Q 

where the root of t" is labelled by A and where the notation txfo/X) in- 
dicates the result of substituting the tree t<i for a particular leaf node of t\ 
which is labelled by X. In this derivation step the adjunction constraint of 
the foot node in t\ replaces the constraint on the root of t" . 

Definition 12. Suppose G = (V, N, S, I, A) is a tag. The tree language 

T(G) generated by G is defined as 

T(G) = {t\ti =4> t for some tj G I and t has no obligatory node labels} 
The string language, L(G), generated by G is defined as 

L{G) = {frontier(t)\t G T(G) and frontier(t) eV*} 

The following theorem shows how to construct a weakly equivalent tag 
G' for a given cftg G with contains only nullary and monadic nonterminals 
and vice versa. 

Theorem 2. The classes of string languages generated by tree adjoining 
grammars and by cftg's with nonterminals of arity at most one and linear 
production rules are the same. 

Remark 1. The linearity constraint mirrors the requirement of a distin- 
guished foot node in the auxiliary trees. 

Proof. Let G = (V, N, S, I, A) be a tag. We construct an equivalent cftg 
G' = S', P), where we let S = V and S n = {A n \A G N and. .4 labels 

a node in I or A with n daughters}, F = {A'\A G N} and T\ = {A\A G N} 
and we let P consist of the following productions: 

S' -» t* for U G I 

A(x) — > t* a {x/A) for t a G A with A G N labelling root and foot node, and 
carrying the null adjunction constraint at the foot node 
A(x) -> t* a {A(x)/A) otherwise 

The trees t* are the result of replacing every non-foot node with a nonter- 
minal label A carrying the obligatory adjunction constraint by the branch 
consisting of the node labelled A immediately dominating the node labelled 
A and of replacing every nonterminal non-foot leaf label A. In the construc- 
tion of the rule set P we have suppressed the numerical subscripts of the 
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terminal symbols originating from the alphabet N. A straightforward, but 
tedious inductive argument will show that G' is weakly equivalent with G 
and even generates the same tree language. 

Suppose now that G = (T,,F,S,P) is a cftg satisfying the constraints 
in the statement of the theorem. An equivalent tag G' = (V, N, S', I, A) is 
obtained in the following manner. Let V = So and let N = U n >o S n U.Fx P. 
We assume that S' is a new start symbol that does not occur in S or T. 
I consists of the result of performing the following operations on the right- 
hand sides of the nonterminal S in P: Occurrences of nonterminal labels F 
are replaced by bared pairs (F, p) such that p is an element in P rewriting 
F. The results of these substitutions are extended by a new root labelled S'. 
The set of auxiliary trees A is obtained by the same substitution operation 
performed this time on the right-hand sides of nonterminals other than S. 
The elements in A, too, are extended by a new root which consists of the 
unbarred pair (F,p) of the replaced nonterminal F and the relevant rule 
label p that licenses the particular replacement of F by the daughter tree 
of the new root label. That very same root label is also substituted for 
the variable x. These new root and foot labels do no real work. They are 
introduced for the sole purpose of complying with the specifications of a 
tree adjoining grammar. The new grammar G' is again, apart from minor 
notational modifications, identical to its "source" and it should be obvious 
that both generate the same string language. 

□ 

5. Conclusion 

As mentioned in the introduction, one of our motivations for looking into 
the relationship between tree adjoining and context-free tree grammars has 
been the intimate connection between descriptive complexity theory and al- 
gebraic language theory. It has been known for nearly 30 years that the 
regular tree languages — and therefore the context-free string languages by 
the fact cited above — are exactly those languages that are definable within 
the weak monadic second-order logic with multiple successors. These defin- 
ability results can easily be extended to context-free tree languages by lifting 
their alphabets and eliminating in this process all their nonterminal symbols 
of arity greater than zero. Given the close relationship between tag's and 
monadic context-free tree grammars, established in this note, it is possible 
to apply the definitional resources of the weak monadic second-order logic to 
tag's and to give grammar independent characterizations of this formalism. 

As another result of the equivalence between tag's and context-free tree 
grammars, parsing techniques and special concepts of finite automata that 
were developed for cftg's, become immmediately applicable to tag's. It re- 
mains to be investigated whether these methods from a different type of 
grammatical formalism can advance the theory of tree adjoining grammars. 
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